This invention relates generally to resource management for storage systems, and in particular to adaptive management of resources shared by multiple consumers.
Virtualization technologies allow hardware resources to be used and shared by multiple consumers. A consumer can be a process running on a computer system that accesses resources to perform certain tasks. An example of consumer is a task related to database operations on a system hosting databases, for example, query processing, data manipulations, reporting, replication, backup, restore, or export. These tasks can require significant amount of system resources. An example of a shared hardware resource is network resource that allows consumers to communicate with external systems. Another example is a bandwidth of storage subsystem. Shared resources are allocated between various consumers. The allocation of resources to individual consumers determines the overall utilization of the hardware resources in a system.
Consumers of resources may be associated with priorities based on the consumer's importance to an end user. For example, certain consumers perform tasks that have higher priority than other consumers or have tighter service level agreements (SLA) requirements. Allocation of hardware resources between consumers need to consider their priorities. Allocations aim to ensure that higher priority tasks get a larger share of resources than lower priority tasks. However, even a low priority task should be able to make progress over time, although its progress may be slow compared to high priority tasks. Improper allocation of resources to consumers may result in starvation of some consumers and hoarding of resources by other consumers. Starvation of a consumer results when the consumer is perpetually denied resources that it needs.
Various strategies are utilized to share resources between consumers. A fixed resource allocation strategy can allocate fixed amount of resources to different consumers based on their priorities. In many cases these fixed amounts are determined upfront or are results of explicit operator input. Fixed resource allocation strategies may not be able to automatically adjust to dynamic changes in consumer needs. A proportional fairness based resource allocation strategy allocates an amount of resources for each consumer proportionate to its anticipated resource consumption. Another resource allocation strategy is a round robin strategy that iterates through consumers in a round robin fashion to allocate resources. Other types of allocation strategies include first come first served type of allocation, fair queuing (max-min fairness) and weighted queuing.